Aiming at the problem of inaccurate matching of weak texture and pure color region in stereo matching and long time consumption of image segmentation algorithms, a stereo matching algorithm fused with image segmentation was proposed. Firstly, the initial image was filtered by Gaussian and smoothed by Sobel to obtain the edge feature map of the image. Secondly, the red, green and blue channel values of the original image were dichotomized by using the Otsu method and then refused to obtain the segmentation template map. Finally, the obtained grayscale map, edge feature map and segmentation template map were applied in the parallax calculation and parallax optimization process in order to calculate the parallax map. The proposed algorithm has the accuracy improved by 14.23 percentage points on average with the time cost per 10 000 pixels increased by 7.16 ms in comparison with Sum of Absolute Differences (SAD) algorithm. The experimental results show that the proposed algorithm can obtain smoother matching results in pure color and weak texture regions and parallax discontinuity regions, and it can automatically calculate the threshold and segment the image faster.